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Abstract 

This  non-experimental,  longitudinal  study  examines  Local  Area  Network  (LAN)  end  user 
satisfaction  (BUS)  among  47  healthcare  executives  at  the  VA  Puget  Sound  Health  Care  System 
(VA  PSHCS).  The  executives  included  in  this  study  were  using  the  LAN  before  October  1, 1997, 
and  after  the  implementation  (February  18, 1998)  of  the  LAN  infi'astructure  enhancements.  This 
study  uses  satisfaction  as  one  measure  to  support  the  LAN  enhancements,  and  as  an  indicator  of 
LAN  success  or  effectiveness  using  the  short  form  instrument  developed  by  Blake  Ives, 
Margrethe  Olsen  and  Jack  Baroudi  in  1983,  and  refined  by  Jack  Baroudi  and  Wanda  Orlikowski 
in  1988.  While  the  VA  PSHCS  improves  their  information  technologies  (IT),  the  organization 
must  balance  quality,  access,  and  cost  of  providing  healthcare  services  to  eligible  veterans. 
Additionally,  the  organization  must  scan  the  horizon  for  opportunities  to  remain  solvent  in  an  era 
of  limited  resources  and  alternative  healthcare  delivery  models.  Therefore,  with  limited 
resources,  healthcare  organizations  that  improve  their  IT  must  somehow  demonstrate  the  added 
value  of  IT  expenditures  and  establish  system  success.  This  paper  presents  three  hypotheses  that 
were  tested  using  the  short  form  instrument  to  measure  LAN  BUS.  The  alternate  hypotheses 
include:  1)  the  short  form  instrument  used  at  the  VA  PSHCS  is  a  reliable  and  valid  measure  of 
LAN  BUS;  2)  healthcare  executive  LAN  BUS  at  the  VA  PSHCS  is  expected  to  increase  over 
time;  and  3)  healthcare  executive  LAN  BUS  at  the  VA  PSHCS  is  independent  of  grouped 
differences  (i.e.,  gender,  age,  position  held  in  the  organization,  duration  of  LAN  usage,  percent 
of  time  using  LAN,  and  LAN  training).  In  each  case,  the  null  hypothesis  was  rejected. 
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Introduction 

The  VA  Puget  Sound  Health  Care  System  (VA  PSHCS)  provides  tertiary  care  services  to 
the  veteran  population  in  the  Pacific  Northwest  United  States.  The  organization  is  one  of  173 
healthcare  facilities  and  systems  (VA  Fact  Sheet,  January  1998)  within  the  Department  of 
Veterans  Affairs’  (VA),  Veterans  Health  Administration  (VHA).  The  VHA  has  22  Veterans 
Integrated  Service  Networks  (VISN;  Kizer,  December  9,  1997a;  VA  Fact  Sheet,  January  1998) 
operating  in  the  United  States  including  the  District  of  Columbia,  and  Puerto  Rico,  Guam,  the 
Philippines,  and  the  U.S.  Virgin  Islands  (Kizer,  1997b). 

The  VHA  has  four  missions: 

1.  "...  to  provide  healthcare  to  eligible  veterans  (Kizer,  1997b,  p.  286)." 

2.  "...  to  conduct  education  and  training  programs  that  enhance  the  quality  of  care  provided  to 
veterans  (Kizer,  1997b,  p.  287)." 

3.  "...  to  conduct  research  that  will  enhance  VA's  provision  of  healthcare  to  veterans  (Kizer, 
1997b,  p.  287)." 

4.  "...  to  serve  as  the  primary  contingency  back-up  to  the  Department  of  Defense  medical  care 
system  and  to  assist  the  PHS  [Public  Health  Service]  and  the  National  Disaster  Medical 
System  [by  providing]  emergency  medical  care  to  victims  of  natural  and  other  disasters 
(Kizer,  1997b,  p.  288)." 

These  missions  lay  the  foundation  for  each  facility  within  the  VHA  to  develop  a  congruent, 

yet  geographically  distinct  strategic  direction.  The  VA  PSHCS's  mission,  vision,  and  values  are: 

MISSION  -  We  proudly  serve  the  Veterans  of  the  Northwest,  by  delivering  the  highest 
quality  health  care  and  services  supported  by  our  education  and  research  programs. 

VISION  -  Together,  we  can  exceed  customer  expectations  by  continually  developing  better 
ways  to  provide  the  highest  quality  health  care  and  services  to  our  veterans  within  an 
environment  that  empowers  and  inspires  excellence. 

VALUES  -  Quality,  Courtesy,  Respect,  Caring,  Integrity,  Commitment,  Teamwork 
(VA  PSHCS  VISTA,  October  28,  1997). 
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These  living  statements  and  qualities  are  the  foxmdation  that  supports  the  rich  tapestry  of  a 
healthcare  organization.  At  the  VA  PSHCS,  the  staff  provides  healthcare  and  support  services  at 
two  divisions  approximately  45  miles  apart.  In  fiscal  year  1997,  the  organization's  bed  services 
included  367  acute  care,  146  extended  care,  54  psychiatric  residential  rehabilitation  treatment, 
and  50  domiciliary  beds  with  a  complement  of 2,574  full-time  employee  equivalents  and  a  $225 
million  medical  care  budget  (VA  PSHCS  Executive  Summary,  1997).  The  organization  also 
realized  a  budget  of  over  $20  million  (VA  and  non-VA  funding)  for  biomedical,  health  services, 
geriatric,  VA/DOD  collaborative,  and  rehabilitation  research  activities.  Additionally,  the 
organization  supported  educational  efforts  through  the  medical  school  affiliation  with  the 
University  of  Washington  and  other  accredited  higher  learning  institutions. 

To  support  the  VA  PSHCS's  mission,  the  organization  is  migrating  fi-om  a  mainframe 
system  to  a  client/server  system  operating  on  a  Local  Area  Network  (LAN)  platform  to  better 
meet  the  needs  of  the  facility's  end  users  (Acting  Chief  IRMS,  personal  communication,  October 
16,  1997).  A  mainframe  system  is  "a  very  large  and  expensive  computer  capable  of  supporting 
hundreds,  or  even  thousands,  of  users  simultaneously  (pcwebopedia,  November  8, 1997). 
Conversely,  a  LAN  is  a  system  of  electronics  (e.g.,  printers,  computers,  and  data  storage  devices) 
connected  together  by  cable  (e.g..  Cat  5  and  fiber  optic)  in  a  small  geographical  area,  which 
allow  users  to  optimize  communication  channels,  such  as  sharing  data  and  information  (Ball,  et 
al,  1995;  Nath,  1990;  Tan,  1995). 

Within  the  organization,  the  Information  Resources  Management  Service  (IRMS)  provides 
the  LAN  infrastructure,  software  and  educational  support  to  the  organization's  LAN  end  users 
(VA  PSHCS  Memorandum  IM-01, 1995;  VA  PSHCS  Memorandum  IM-22, 1995).  The  facility 
has  a  LAN  operating  at  each  Division  (Seattle  and  American  Lake;  Acting  Chief  IRMS,  personal 
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communication,  October  16, 1997).  The  two  LANs  are  connected  via  two  T-1  lines  that  transmit 
data  between  the  two  divisions.  The  LAN  infrastructure  at  the  Seattle  Division  used  a  hub 
environment  and  the  American  Lake  Division's  LAN  used  an  TUT-box  environment  before  the 
enhancements.  During  November  1997  through  mid-February  1998,  the  IRMS  staff  enhanced 
the  LAN  infrastructure  in  several  critical  ways:  1)  replacing  the  hubs  and  TUT-boxes  with 
switches;  2)  installing  Category  5  cable  and  a  fiber  optic  backbone;  and  3)  providing 
enhancements  at  the  desktop  such  as  installing  64MB  (Acting  Chief  IRMS,  personal 
communication,  October  16, 1997  and  April  10, 1998). 

While  the  VA  PSHCS  improves  their  information  technologies  (IT),  the  organization  must 
balance  quality,  access,  and  cost  of  providing  healthcare  services  to  eligible  veterans  within  the 
local  community  and  the  VISN.  Additionally,  the  organization  must  scan  the  horizon  for 
opportunities  to  remain  solvent  in  an  era  of  limited  resources,  restriction  of  growth,  and 
alternative  financing  and  healthcare  delivery  methods  (Williams  &  Torrens,  1993).  Therefore, 
with  limited  resources,  healthcare  organizations  that  improve  their  IT  must  somehow 
demonstrate  added  value  of  IT  expenditures  at  least  in  terms  of  opportunity  cost  (e.g.,  forgoing 
additional  clinical  staff)  and  establish  system  success  (e.g.,  effectiveness);  this  paper  examines 
the  latter  in  terms  of  end  user  satisfaction  (EUS). 

Problem  Statement 

According  to  Blake  Ives,  Margrethe  Olsen  and  Jack  Baroudi  (1983),  EUS  is  a  perceived 
measure  that  substitutes  for  objective  criteria  of  information  system  effectiveness  where  few 
objective  measures  are  available.  The  authors  state  that  the  net  value  of  an  information  system 
equals  the  actual  benefits  (e.g.,  improved  effectiveness)  minus  the  development  and  operational 
costs.  However,  measuring  the  actual  benefits  are  challenging  at  best. 
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According  to  many  authors,  EUS  with  a  system  such  as  a  LAN,  is  the  critical  factor  that 
defines  success  for  the  system  (Baliey  &  Pearson,  1983;  Baroudi  &  Orlikowski,  1988;  Gatian, 
1994;  Guimaraes  &  Igbaria,  1997;  Ives,  Olsen  &  Baroudi,  1983;  Melone,  1990;  Nath,  1990; 
Ryker,  Nath  &  Henson,  1997).  On  the  basis  of  prior  research,  EUS  was  used  in  this  study  as  a 
measure  to  support  the  enhancements  to  the  LAN  and  as  an  indicator  of  LAN  success  or 
effectiveness  as  the  VA  PSHCS  does  not  have  LAN  satisfaction  information  fi-om  healthcare 
executives  within  the  organization. 

Literature  Review 

James  Bailey  and  Sammy  Pearson  (1983)  published  their  efforts  to  measure  computer  user 
satisfaction  (CUS).  The  authors  developed  and  analyzed  an  instrument  to  measure  CUS  using 
the  semantic  differential  technique.  As  developed  by  Charles  Osgood,  G.  Suci  and 
P.  Tannenbaum,  semantic  differential  is  a  scaling  method  for  measuring  "psychological 
meanings  of  an  object  to  an  individual  (Bailey  &  Pearson,  1983;  Cooper  &  Emory,  1995)." 
According  to  Donald  Cooper  and  C.  William  Emery  (1995),  this  technique  is  widely  used  for 
business  studies  (e.g.,  organizational,  political,  and  marketing). 

The  CUS  instrument  included  39  scales  (questions)  with  six  bipolar  adjective  pair  items  on 
a  seven  point  interval  scale  (e.g.,  extremely  dissatisfied  to  extremely  satisfied)  to  measure  CUS 
(Bailey  &  Pearson,  1983).  The  authors  measured  content,  construct,  and  predictive  validity.  The 
first  four  items  were  used  to  measure  CUS.  The  fifth  item  was  used  to  establish  internal  validity 
with  the  first  four  items.  The  sixth  item  was  used  to  determine  the  importance  (weight)  of  the 
scale.  Reliability  was  demonstrated  using  an  analysis  of  variance  among  the  first  four  items  of 


the  39  scales. 
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Blake  Ives,  Margrethe  Olsen  and  Jack  Baroudi  (1983)  developed  an  instrument  to  measure 
"user  information  satisfaction  (UIS;  p.  785)."  Essentially,  the  authors  used  Bailey  &  Pearson’s 
(1983)  instrument  to  develop  a  new  13  scale  instrument,  known  as  the  "short  form  (p.  530)." 
Through  a  series  of  reliability  (Cronbach's  alpha)  and  validity  procedures  (content,  predictive, 
and  construct),  Ives,  Olsen  and  Baroudi  (1983)  developed  and  validated  the  short  form.  The 
authors  used  the  process  below  to  develop  the  short  form  instrument. 

The  authors  first  removed  those  scales  with  "undesirable  psychometric  characteristics" 

(p.  791)  fi-om  the  original  39  scales.  Second,  of  the  remaining  scales,  the  authors  removed  those 
scales  with  factor  loading  below  .50.  After  the  second  step,  three  factors  (information  product, 
electronic  data  processing  staff  and  services,  and  knowledge  and  involvement  with  five,  five,  and 
three  scales,  respectfully)  remained  to  comprise  the  13  scale  short  form  instrument.  Third,  the  13 
scales  were  given  two  bipolar  adjective  pair  items.  Fourth,  to  validate  that  the  short  form 
instrument  measured  UIS  as  in  Bailey  and  Pearson's  39  scales,  the  authors  removed  the  13  scales 
on  the  short  form  instrument  from  the  39  scales.  The  short  form  instrument  total  satisfaction  was 
then  correlated  with  the  remaining  scales  (r  =  .90,  p  =  .001).  The  correlation  between  the  short 
form  instrument  total  satisfaction  score  and  the  first  four  items  of  Bailey  and  Pearson's  (1983)  39 
scales  resulted  in  a  correlation  of  .54,  p  =  .001 .  According  to  Ives,  Olsen  and  Baroudi  (1983), 
these  correlations  "provide  substantial  evidence  that  the  short  form  questionnaire  is  a  sound 
general  measure  of  Pearson's  original  UIS  concept  (p.  791)." 

Jack  Baroudi  and  Wanda  Orlikowski  (1988)  used  the  short  form  instrument  that  was 
developed  by  Ives,  Olsen  and  Baroudi  in  1983  to  measure  UIS.  In  Baroudi  and  Orlikowski's 
(1988)  article,  they  analyzed  the  short  form  instrument  and  provided  recommendations  for  future 
use  of  the  instrument.  The  authors  used  the  short  form  instrument  to  measure  UIS  among  26 
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New  York  City  organizations,  none  of  which  were  healthcare  facilities  or  systems.  Three- 
hundred-and-fifty-eight  employees  responded  to  the  survey.  The  respondents  were  asked  to 
remain  anonymous  however  most  were  clerical  or  support  personnel.  Their  total  user 
satisfaction  score  was  determined  by  summing  the  scores  from  the  short  form  instrument's  13 
scales.  In  addition,  the  satisfaction  scores  for  the  three  factors  (information  product,  electronic 
data  processing  staff  and  services,  and  knowledge  and  involvement)  were  calculated  by 
averaging  the  respective  scale  scores.  The  averaging  process  was  used  to  measure  across  the 
three  factors.  Thus,  the  total  satisfaction  score  can  range  from  -39  to  39  and  the  scores  for  the 
three  factors  can  range  from  -3  to  3. 

According  to  Baroudi  and  Orlikowski  (1988),  the  short  form  construct  and  convergent 
validity  measures  were  met.  Additionally,  the  authors  concluded  that  previous  evaluations  of  the 
short  form  provided  "extensive  evidence"  (p.  47)  of  content  validity. 

Construct  validity  was  demonstrated  using  the  correlation  between  each  of  the  13  scales 
and  the  total  UIS  score  (correlations  ranged  from  .35  to  .69  with  11  scales  over  .50,  p  <  .001). 
This  is  the  weaker  method  when  compared  to  factor  analysis,  according  to  the  authors.  The 
authors  then  used  the  method  of  factor  analysis  with  "varimax  rotation"  (p.  48)  and  an  "eigen 
value  of  1  (p.  48)."  After  five  iterations,  the  factor  analysis  (information  product,  electronic  data 
processing  staff  and  services,  and  knowledge  and  involvement)  solution  accounted  for  "68%  of 
total  variation  (p.  48)."  With  this  outcome,  the  authors  suggest  that  the  three  factors  are  "stable 
and  provide  strong  evidence  for  construct  validity  (p.  48)." 

Convergent  validity  was  determined  through  an  interview  process  among  users  in  five 
organizations.  The  users  in  two  organizations  were  "generally"  (p.  48)  satisfied  with  their 
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information  systems,  and  users  in  the  remaining  three  organizations  were  "very  dissatisfied" 

(p.  48)  with  their  information  systems.  The  users  in  the  five  organizations  were  asked  to 
complete  the  short  form  instrument  and  then  the  mean  scores  were  computed  for  each  group 
(n  =  26,  satisfied  mean  =  14.5;  n  =  44,  dissatisfied  mean  =  -5.1).  The  authors  then  computed  a 
t-test  between  the  two  groups  (significant  at  p  <  .001). 

The  authors  used  Cronbach's  alpha  to  demonstrate  the  reliability  of  the  short  form 
instrument.  To  reduce  the  tendency  of  users  to  mark  straight  down  the  form,  Baroudi  and 
Orlikowski  (1988)  reverse  ordered  some  of  the  scales  (e.g.  positive  on  right  and  negative  on  left; 
and  positive  on  left  and  negative  on  right).  Cronbach's  alpha  for  the  13  scales'  two  item  bipolar 
adjective  pairs  was  above  .80  for  all  item  pairs;  total  satisfaction  and  the  13  scales  was  at  .89; 
EDP  staff  and  services  was  at  .83;  information  product  was  at  .89;  and  knowledge  and 
involvement  was  at  .72.  Baroudi  and  Orlikowski  (1988)  then  concluded  that  the  short  form  is 
"internally  consistent  and  thus  reasonably  free  of  measurement  error  (p.  50)."  On  the  basis  of 
Baroudi  and  Orlikowski’s  (1988)  findings,  the  short  form  instrument  was  selected  to  measure 
LAN  EUS  at  the  VA  PSHCS. 

Purpose  Statement 

The  purpose  of  this  study  is  to  examine  LAN  EUS  among  the  healthcare  executives  at  the 
VA  PSHCS  who  were  using  the  LAN  before  October  1, 1997,  and  after  the  implementation 
(February  18, 1998)  of  the  LAN  enhancements.  A  presentation  follows  with  the  identification  of 
the  three  LAN  EUS  hypotheses  that  were  tested  during  this  study. 
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Hypothesis  Formulation 

Reliability  and  Validity  of  the  Short  Form  Instrument 

Previous  researchers  as  noted  in  the  literature  review  established  reliability  and  validity  of 
the  satisfaction  instrument.  Therefore,  the  author  will  also  measure  the  reliability  and  validity  of 
the  short  form  instrument  used  in  this  study. 

Hypothesis  1:  The  short  form  instrument  used  at  the  VA  PSHCS  is  a  reliable  and 
valid  measure  of  LAN  EUS. 


End  User  Satisfaction 

This  section  of  hypothesis  development  includes  total  LAN  EUS  and  the  three  factors  that 
comprise  68%  of  EUS  variation  (Baroudi  and  Orlikowski,  1988).  These  satisfaction  levels  are 
expected  to  increase  as  the  LAN  is  enhanced  to  indicate  system  success  or  effectiveness.  The 
total  EUS  score  is  the  overall  satisfaction  with  the  LAN.  The  three  factors  are  defined  below. 

1 .  The  LAN  information  product  "factor  is  the  respondent's  self-reported  assessment  of  the 
quality  of  output  delivered  by  the  [LAN]  information  system  (Baroudi  and  Orlikowski,  1988, 
p.48).” 

2.  The  IRMS  staff  and  services  "factor  is  the  respondent's  self-reported  assessment  of  the 
attitude  and  responsiveness  of  the  EDP  [IRMS]  staff  as  well  as  the  quality  of  their 
relationship  with  the  EDP  [IRMS]  staff  (Baroudi  and  Orlikowski,  1988,  p.  48).” 

3.  The  LAN  knowledge  and  involvement  "factor  is  the  respondent's  self-reported  assessment  of 
the  quality  of  [LAN]  training  provided,  their  understanding  of  the  [LAN]  system,  and  their 
participation  in  its  [the  LANs]  development  (Baroudi  and  Orlikowski,  1988,  p.  48).” 


Hypothesis  2;  Healthcare  executive  LAN  EUS  at  the  VA  PSHCS  is  expected  to 
increase  over  time. 
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EUS  and  Grouped  Differences 

The  following  section  presents  EUS  and  grouped  differences  (i.e.,  gender,  age,  position 
held  in  the  organization,  duration  of  LAN  usage,  percent  of  time  using  the  LAN,  and  LAN 
training)  as  variables  for  the  third  hypothesis. 

EUS  and  Age 

In  addition  to  the  13  scales  provided  in  the  short  form  instrument,  certain  demographic 
information  was  used  in  this  study  such  as  the  position  held  in  the  organization,  the  age  and  the 
gender  of  the  respondents  (Bowman,  et  al,  1993;  Guimaraes  &  Igbaria,  1997;  Nath  1990).  This 
information  should  provide  further  insight  to  the  cohort  even  with  the  small  sample  size  (if  all  47 
people  respond  to  the  survey  and  there  is  parity  between  gender,  the  small  sample  size  of  23  and 
24  would  be  below  the  minimum  sample  size  for  significance,  n  =  30).  This  example  is  true  for 
the  variable  position  held  in  the  organization  as  well.  However,  the  age  variable  has  the  potential 
to  reach  a  sample  size  equal  to  or  greater  than  30. 

EUS  and  Duration  of  LAN  Usage 

Other  elements  of  end  user  characteristics  (LAN  usage  in  monthly  duration,  intensity  by 
percent  of  LAN  time  usage,  and  LAN  training)  were  used  during  previous  information  system 
user  satisfaction  studies  (Baroudi,  Olsen  and  Ives,  1986;  Guimaraes  &  Igbaria,  1997;  Nath,  1989; 
Nath  1990;  Smith,  1993).  According  to  Harold  Smith  (1993)  in  his  article  entitled,  "Local  Area 
Network  User  Satisfaction:  An  Analysis  of  Relevant  Factors  (p.  15),"  the  length  of  time  the  LAN 
is  used  (duration  in  months)  influences  EUS. 
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BUS  and  Percent  of  Time  Using  the  LAN 

According  to  Ravinder  Nath  (1989)  in  his  article  entitled,  "Are  Frequent  Users  More 
Satisfied  (p.  557),"  the  percentage  of  time  spent  using  a  system  is  a  significant  determinant  of 
user  satisfaction.  Other  studies  have  shown  a  correlation  between  user  satisfaction  and  system 
usage  (Baroudi,  Olsen  &  Ives,  1986;  Guimaraes  &  Igbaria,  1997). 

BUS  and  LAN  Training 

User  training  has  also  shown  to  have  a  positive  relationship  (correlation)  with  user 
satisfaction  of  a  system  (Guimaraes  &  Igbaria,  1997;  Nath,  1990).  According  to  Tor  Guimaraes 
and  Magid  Igbaria  (1997),  user  training  is  a  key  component  in  user  satisfaction  as  well  as  in 
promoting  system  usage.  Additionally,  Nath  (1990)  found  similar  results  from  his  study  entitled, 
"The  Impact  of  Local  Area  Networks  on  Users  and  Their  Work:  A  Field  Study  (p.  15)."  He 
found  that  there  is  a  "positive  association  between  user-training  and  their  satisfaction  with  the 
network  (p.  22)." 

Hypothesis  3:  Healthcare  executive  LAN  EUS  at  the  VA  PSHCS  is  independent  of 
grouped  differences. 

Methods  and  Procedures 

Based  on  the  evidence  from  Baroudi  and  Orlikowski  (1988),  the  short  form  instrument  was 
used  to  measure  healthcare  executive  LAN  EUS  at  the  VA  PSHCS.  The  following  six 
modifications  were  made  to  the  original  short  form  instrument  to  better  relate  to  the 
organization's  healthcare  executives: 

1 .  The  phrase  electronic  data  processing  (EDP)  was  replaced  with  Information  Resources 
Management  Service  (IRMS). 

2.  The  acronym  LAN  was  added  to  the  information  product  factor  scales. 
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3.  The  definition  of  LAN  output  (Smith,  1993)  is  provided  on  the  survey  tool  in  place  of  the 
original  computer  support  definition. 

4.  Certain  demographic  questions  were  added  to  the  survey  instrument,  such  as  the  position 
held  in  the  organization,  age  and  gender. 

5.  A  section  was  added  at  the  end  of  the  survey  tool  for  respondent  anonymity  (Galletta  & 
Lederer,  1989). 

6.  The  name,  the  intra-organizational  address  for  returning  the  completed  instrument  and  the 
telephone  extension  of  the  author  was  provided  on  the  survey  instrument. 

This  graduate  management  project  was  a  non-experimental,  longitudinal  study  of  47 
healthcare  executives'  satisfaction  (11  Administrative  Officers  [AO],  24  Service 
Chief/Assistant/Section  Chiefs  [SASC],  and  12  Senior  Management/Staff  [SMS])  with  the 
organization's  LAN.  These  executives  were  using  the  LAN  as  of  October  1, 1997,  and  were 
asked  to  respond  to  a  LAN  BUS  instrument  that  was  distributed  twice-November  6, 1997,  and 
February  26, 1998.  A  standardized  memorandum  accompanied  the  instrument,  which  informed 
the  cohort  that  the  study  was  voluntary  and  anonymous.  The  executives  were  identified  through 
the  following  two  step  procedure:  1)  the  services  and  programs  connected  to  the  LAN  as  of 
October  1,  1997,  were  identified  fi'om  the  IRMS  leadership;  and  2)  the  healthcare  executives 
working  in  those  services  and  programs  were  identified  fi-om  the  electronic  telephone  directory 
in  the  VA  PSHCS’s  VISTA  computer  system. 

To  date,  only  one  identified  research  study  using  repeated  measures  was  available  for 
comparison  using  the  short  form  instrument.  Dennis  Galletta  and  Albert  Lederer  (1989)  advised 
in  their  repeated  measure  study  to  use  caution  with  parametric  statistical  procedures  for  ordinal 
scales.  Conversely,  Donald  Cooper  and  C.  William  Bmory  (1995)  state  that  the  use  of 
parametric  statistical  analysis  for  ordinal  scales  is  commonly  used  in  research  efforts  as  the 
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ordinal  scales  approach  interval  scales.  Moreover,  the  short  form  instrument  uses  the  semantic 
differential  technique,  which  uses  interval  scales.  Therefore,  parametric  tests  were  used  in  this 
study. 

Reliability 

Reliability  is  the  "accuracy  or  precision  of  a  measuring  instrument  (Kerlinger,  1986, 
p.  405)."  More  importantly,  reliability  is  "a  contributor  to  validity  and  is  a  necessary  but  not 
sufficient  condition  for  validity  (Cooper  &  Emory,  1995,  p.  153)."  The  instrument's  reliability 
was  demonstrated  using  Cronbach's  alpha  for  the  two  bipolar  adjective  pair  items  for  each  of  the 
13  scales,  for  the  total  satisfaction  score  and  the  13  scales,  and  for  each  of  the  three  factors  and 
their  respective  scales.  This  was  accomplished  for  both  cross-sectional  surveys. 

Validity 

According  to  Fred  Kerlinger  (1986),  validity  can  best  be  defined  by  asking  the  question, 
"are  we  measuring  what  we  think  we  are  measuring  (p.  417)?”  As  previously  stated,  three  types 
of  validity  were  used  to  validate  the  short  form  instrument  (content,  construct  and  convergent 
validity).  Content  validity  for  the  instrument  has  been  demonstrated  by  previous  researchers  and 
will  be  assumed  for  the  purpose  of  this  study.  Construct  validity  was  determined  by  Pearson's 
correlation  coefficient  (r)  between  the  total  EUS  and  each  of  the  13  scales.  Construct  validity 
was  further  demonstrated  in  previous  research  (Baroudi  &  Orlikowski,  1988)  using  factor 
analysis  with  varimax  rotation  and  was  not  repeated  in  this  study.  A  paired-samples  t-test  was 
used  to  measure  convergent  validity  between  the  two  surveys'  total  satisfaction  scores. 
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To  determine  whether  to  reject  or  accept  the  null  hypotheses  one  through  three,  the 
following  statistical  methods  were  conducted  for  the  appropriate  hypothesis.  In  addition  to  the 
methodologies  below,  descriptive  statistics  were  calculated  for  organizational  added  value. 

Hai :  Use  Cronbach's  alpha  for  reliability;  Pearson's  correlation  coefficient  (r)  for  construct 
validity;  and  a  paired-samples  t-test  for  convergent  validity  as  described  in  the  reliability  and 
validity  paragraphs  in  the  methodology  section  above. 

Ha2:  Use  the  BUS  scores  (total  and  three  factors)  from  surveys  1  and  2. 

Has:  Use  Pearson's  correlation  coefficient  (r)  to  identify  significant  grouped  differences  with  the 
satisfaction  scores. 

Bthics 

The  views  and/or  opinions  expressed  in  this  paper,  unless  otherwise  stated,  are  those  of  the 
author  and  do  not  reflect  the  official  policy  of  the  Department  of  Veterans  Affairs,  the  Veterans 
Health  Administration,  the  VA  PSHCS,  and/or  the  U.S  Army-Baylor  University  Program  in 
Health  Care  Administration.  The  data  that  was  gathered  for  this  study  was  necessary  to  achieve 
four  objectives: 

1 .  To  measure  LAN  BUS. 

2.  To  identify  opportunities  for  LAN  end  user  training. 

3.  To  measure  the  IRMS  staff  relations  with  LAN  end  users. 

4.  To  meet  the  U.S.  Army-Baylor  University  Graduate  Program  in  Health  Care  Administration's 
graduate  management  project  requirement. 

Copyright  Permission 

Once  a  survey  instrument  is  published  in  a  journal,  the  author  usually  relinquishes  the 
copyright  to  the  publisher  (K.  Finstuen,  personal  communication,  October  27, 1997).  Bven 
though  Baroudi  and  Orlikowski  (1988)  included  the  survey  tool  in  their  paper,  every  effort  was 
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made  to  obtain  permission  from  the  authors  to  use  the  short  form  instrument  for  this  study.  An 
electronic-mail  (e-mail)  message  was  sent  to  Jack  Baroudi,  Ph.D.,  seeking  permission  to  use  the 
short  form  instrument  for  this  study.  On  October  29, 1997,  Dr.  Baroudi  responded  via  an  e-mail 
message  granting  permission  to  use  the  short  form  instrument  (J.  Baroudi,  personal 
communication,  October  29, 1997). 

In  addition,  a  request  to  use  the  short  form  instrument  from  Baroudi  and  Orlikowski's  1988 
article  was  sent  to  the  publisher  (M.E.  Sharpe,  Incorporated)  on  October  28, 1997.  The  publisher 
replied  via  a  facsimile  on  October  29,  1997,  granting  permission  to  use  the  short  form  instrument 
for  this  study;  the  copyright  permission  follows  below: 

Jack  J.  Baroudi  and  Wanda  J.  Orlikowski,  A  Short-Form  Measure  of  User  Information 
Satisfaction:  A  Psychometric  Evaluation  and  Notes  on  Use,  Journal  of  Management 
Information  Systems,  Vol.  4,  No.  4,  Spring  1988,  pp.  44-59,  including  Survey.  Please  note 
that  this  permission  is  for  a  one-time  use  only  and  that  you  must  reapply  for  future  uses 
and/or  semester  classes  (G.  Chandoha,  personal  communication,  October  29, 1997). 

Results 

This  study  was  a  non-experimental,  longitudinal  study  of  47  healthcare  executives’  LAN 
EUS  at  the  VA  PSHCS.  The  instrument  was  distributed  on  November  6,  1998,  to  measure  EUS 
prior  to  the  LAN  enhancements.  The  same  survey  was  distributed  on  February  26, 1998,  to 
obtain  post-LAN  enhancement  EUS  data.  Of  the  47  surveys  distributed  to  the  cohort,  3 1 
(survey  1)  and  30  (survey  2)  executives  completed  the  short  form  as  shown  in  Table  1 .  The 
respondents'  demographic  information  is  shown  in  Table  2. 
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Table  1 

Response  Rates 


Overall  Response  Rates 


Response  Rates  by  Management  Level 


Survey  Number  in  Number  Percent  Administrative  Service/AsstVSection  Senior 

Sample  Returned  Response  Officer  Chief  Mgt/Staff 


(#/n) 

(%) 

(#/n) 

(%) 

(#/n) 

(%) 

1 

47 

31* 

66% 

11/11 

100% 

14/24 

58% 

6/12 

50% 

2 

47 

30** 

64% 

9/11 

82% 

15/24 

63% 

6/12 

50% 

Note.  *  =  37  total  responses  minus  6  incomplete  responses;  **  =  35  total  responses  minus  5  incomplete  responses. 
(#/n)  =  number  of  responses  by  management  level  /  number  of  total  surveys  distributed  by  management  level. 

(%)  =  response  rate  by  management  level. 

The  EUS  score  was  determined  by  summing  the  scores  from  the  short  form  instrument's  13 
scales.  In  addition,  the  three  factors'  (LAN  information  product,  IRMS  staff  and  services,  and 
user's  LAN  knowledge  and  involvement)  satisfaction  scores  were  determined  by  summing  and 
averaging  the  respective  scales.  The  averaging  process  was  used  to  measure  across  the  three 
factors.  Thus,  the  total  satisfaction  score  can  range  from  -39  to  39  and  the  satisfaction  scores  for 
the  three  factors  can  range  from  -3  to  3  as  shown  in  Table  3. 

Reliability 

The  reliability  of  the  short  form  instrument  was  demonstrated  using  Cronbach's  alpha  for 
the  two  bipolar  adjective  pair  items  for  each  of  the  13  scales  as  shown  in  Table  4;  for  the  total 
satisfaction  score  and  the  13  scales  (survey  1,  .77;  survey  2,  .77);  and  the  three  factors.  The 
IRMS  staff  and  services  factor  had  a  reliability  index  of  .89  for  survey  1  and  .91  for  survey  2. 
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The  LAN  information  product  factor  had  a  reliability  index  of  .87  for  survey  1  and  .92  for 
survey  2.  The  user's  LAN  knowledge  and  involvement  factor  had  a  reliability  index  of  .91  for 
survey  1  and  .93  for  survey  2. 

Table  2 

Respondents’  Demographic  and  Other  Mean  LAN  Values 


Management  Demographic  Values 


Item  Total  Survey  Administrative  Service/Asst./Section  Senior 

Values*  Officer**  Chief'*  Mgt/Staff* 

(SI)  (S2)  (SI)  (S2)  (SI)  (S2)  (SI)  (S2) 


Male 

61% 

63% 

55% 

56% 

71% 

73% 

50% 

50% 

Female 

39% 

37% 

45% 

44% 

29% 

27% 

50% 

50% 

Age 

46 

46 

45 

47 

46 

45 

48 

50 

Months  using 

LAN 

13 

11 

12 

10 

13 

11 

16 

14 

Percent  of  time 
using  LAN 

38% 

47% 

41% 

43% 

36% 

49% 

36% 

48% 

LAN  training 
received 

68% 

70% 

100% 

89% 

57% 

67% 

33% 

50% 

Number  of  days 
trained  before  (-) 
or  after  (+)  LAN 
connection 

+12 

+10 

-4 

+9 

+11 

+7 

+12 

+25 

Note.  Values  represent  the  item  mean  (arithmetic  average). 


SI  =  Survey  1;  S2  =  Survey  2 

*  =  Sample  size  for  each  survey  (S 1 ,  n  =  3 1 ;  S2,  n  =  30) 

**  =  Sample  size  for  Administrative  Officer  (SI,  n  =  1 1;  S2,  n  =  9),  Service/Assistant/Section  Chief  (SI,  n  =  14; 
S2,  n  =  15),  Senior  Management/Staflf  (SI,  n  =  6;  S2,  n  =  6). 
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Validity 

Three  types  of  validity  were  considered  for  this  study  (content,  construct  and  convergent 
validity).  Content  validity  for  the  instrument  was  demonstrated  by  previous  researchers  and  was 
assumed  for  the  purpose  of  this  analysis.  Construct  validity  was  demonstrated  using  Pearson's 


Tables 

Satisfaction  Scores 


Factors 

Survey/Management 

Level 

Total 

Satisfaction 

IRMS  StafP* 
and  Services 

LAN  Information'"* 
Product 

User  Knowledge** 
and  Involvement 

Survey  1 

13.01 

1.04 

1.20 

.52 

Survey  2 

16.19 

1.30 

1.55 

.63 

Administrative  Officer 

Survey  1 

16.27 

1.25 

1.47 

.89 

Survey  2 

19.78 

1.51 

1.70 

1.17 

Service/Assistant/Section  Chief 

Survey  1 

13.46 

1.13 

1.09 

.80 

Survey  2 

16.89 

1.28 

1.67 

.73 

Senior  Management/Staff 

Survey  1 

5.98 

.43 

.95 

-  .81 

Survey  2 

9.08 

1.02 

1.05 

-  .42 

and  the  three  factors  (IRMS  staff  and  services,  LAN  information  product,  and  the  user's  LAN  knowledge  and 
involvement). 

*  =  The  total  satisfaction  score  can  range  from  -39  to  39. 

♦♦  =  The  three  factors'  scores  can  range  from  -3  to  3. 
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correlation  coefficient  (r)  between  the  total  EUS  and  each  of  the  13  scales  (see  Table  5).  A 
paired  samples  t-test  was  used  to  demonstrate  convergent  validity  between  the  two  survey  total 
satisfaction  scores  (see  Table  6). 


Table  4 

Reliability  Scores 


Scales  One  Through  13 

Cronbach’s  alpha 
Survey  1  Survey  2 

1.  Relationship  with  the  IRMS  Staff 

.93 

.98 

2.  Processing  of  requests  for  changes  to  existing  LAN  systems 

.91 

.97 

3 .  Degree  of  LAN  training  provided  to  users 

.92 

.99 

4.  User's  understanding  of  LAN  systems 

.93 

.98 

5.  User's  feeling  of  participation 

.91 

.97 

6.  Attitude  of  the  IRMS  Staff 

.98 

.90 

7.  Reliability  of  LAN  output 

.96 

.92 

8.  Relevancy  of  LAN  output 

.92 

.89 

9.  Accuracy  of  LAN  output 

.97 

.98 

10.  Precision  of  LAN  output 

.83 

.67* 

1 1 .  Communication  with  the  IRMS  staff 

.94 

.97 

12.  Time  required  for  new  LAN  development 

.94 

.96 

13.  Completeness  of  the  LAN  output 

^  -r  1  1  T1  _ 1 _ _ 1  _ _ _ 1 _ a  „ 

.97 

_ _ 1 _ 

.97 

Note.  The  13  scales  have  two  items.  Each  of  the  items  by  scale  was  analyzed  using  Cronbach’s  alpha  to  obtain  a 
reliability  score.  A  score  of  .70  or  above  is  desired  for  most  research  studies,  and  often  time  and  costs  are  wasted  to 
increase  the  index  beyond  .80  (Nunnally,  1978). 


*  =  A  reliability  score  of  .67  was  achieved  with  a  (-3,  3)  outlier  included.  However,  a  reliability  score  of  .96  can  be 
achieved  by  removing  one  outlier  (-3, 3)  from  Survey  2,  scale  10. 
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Table  5 

Scale  Correlations  with  Total  Score 


Scales  One  Through  13 

Pearson's  (  r ) 
Survey  1  Survey  2 

1 .  Relationship  with  the  IRMS  Staff 

.87 

.96 

2.  Processing  of  requests  for  changes  to  existing  LAN  systems 

.83 

.93 

3.  Degree  of  LAN  training  provided  to  users 

.86 

.98 

4.  User's  understanding  of  LAN  systems 

.86 

.96 

5.  User's  feeling  of  participation 

.83 

.95 

6.  Attitude  of  the  IRMS  Staff 

.96 

.82 

7.  Reliability  of  LAN  output 

.92 

.86 

8.  Relevancy  of  LAN  output 

.86 

.82 

9.  Accuracy  of  LAN  output 

.95 

.97 

10.  Precision  of  LAN  output 

.71 

.52* 

1 1.  Communication  with  the  IRMS  staff 

.90 

.94 

12.  Time  required  for  new  LAN  development 

.89 

.93 

13.  Completeness  of  the  LAN  output 

.94 

.95 

Note.  Pearson’s  correlation  coefficient  (r)  was  used  to  measure  the  correlation  between  the  total  satisfaction  and 
each  of  the  13  scale  scores.  The  closer  the  correlation  is  to  ±  1.0,  the  greater  the  correlation;  the  closer  the 
correlation  is  to  0,  the  poorer  the  relationship  between  the  variables.  The  correlations  for  each  survey  are  presented 


in  this  table. 


*  r  >  .456  is  significant  p  <  .001  (survey  2).  Note:  sample  size  for  survey  1,  n  =  31  (29  df)  and  survey  2,  n  =  30 
(28  df). 

*=  Correlation  of  .52  was  achieved  with  one  outlier  (-3, 3)  included  in  survey  2,  scale  10.  Without  the  outlier,  the 
correlation  score  improves  to  .92. 
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Table  6 

Paired  Samples  Test 


Satisfaction 

Survey  1 

November  6, 1997 

Mean  S.D. 

Survey  2 

February  26, 1998 

Mean  S.D. 

t 

P 

IRMS  Staff  and  Services 

1.04 

1.04 

1.30 

.96 

1.276 

.212 

LAN  Information  Product 

1.20 

.96 

1.55 

.88 

2.233 

.033* 

User's  LAN  Information 

.52 

1.44 

.63 

1.47 

.440 

.663 

and  Knowledge 

Total  Satisfaction 

13.01 

12.34 

16.19 

12.20 

1.584 

.124 

Note.  *  =  LAN  Information  Product  is  significant  n  =  3 1  (30  df). 


Discussion  and  Conclusions 

Hypothesis  1 

The  findings  from  this  study  provide  further  evidence  to  support  the  reliability  and  validity 
of  the  short  form  instrument.  These  statistical  methods  are  the  essential  foundation  of  this  study. 
Without  establishing  the  short  form  instrument's  reliability  (does  the  short  form  instrument 
consistently  and  accurately  measure  satisfaction?)  and  validity  (does  the  short  form  instrument 
measure  satisfaction?),  information  derived  from  the  data  might  be  erroneous. 

The  reliability  index  for  all  scales  (survey  1  and  2)  were  above  .80  with  one  exception.  In 
survey  2,  scale  10,  (see  Table  4)  one  subject  responded  at  the  extreme  interval  ends  of  the  two 
items  (-3, 3).  With  this  outlier,  the  reliability  index  is  .67.  However,  with  the  outlier  removed 
from  the  sample,  the  index  improved  to  .96.  According  to  Jum  Nunnally  (1978),  a  score  of 
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.70  or  above  is  desired  for  most  research  studies,  and  often  money  and  time  are  wasted  to 
increase  the  index  beyond  .80.  Therefore,  with  all  but  one  of  the  indexes  above  .80,  the  short 
form  instrument  is  considered  to  be  reasonably  free  of  measurement  error. 

The  validity  methods  for  the  short  form  instrument  included  content,  construct,  and 
convergent  validity.  Content  validity  has  been  established  in  previous  studies  and  was  assumed 
for  this  study.  Construct  validity  was  demonstrated  using  Pearson's  correlation  coefficient  (r) 
where  all  correlations  were  significant  as  shown  in  Table  5.  Twelve  of  the  13  scale  correlations 
(r)  were  greater  than  .82,  with  scale  10,  survey  1  at  .72  (n  =  3 1, 29  df)  and  survey  2  at  .52 
(r  >  .456  is  significant,  p  <  .001,  n  =  30  (28  df;  Edwards,  1984).  Convergent  validity  was 
measured  using  a  paired  samples  t-test  for  the  total  satisfaction  scores  fi-om  each  survey  (see 
Table  6). 

Refinements  to  the  short  form  instrument  in  subsequent  research  included  repeated 
measures  (test-retest)  in  only  one  study  (Galletta  &  Lederer,  1989).  Test-retest  was  provided  in 
Baroudi  and  Orlikowski's  (1988)  article  as  a  suggestion  for  further  research  and  in  Galletta  and 
Lederer's  (1989)  study  as  a  suggestion  for  continued  future  research.  The  correlation  of  the  two 
total  satisfaction  scores  was  significant  (n  =  3 1,  r  =  .58,  p  <  .001,  30  df).  With  all  of  the 
correlations  for  construct  and  convergent  validity  being  significant  and  the  prior  research 
outcomes  for  content  validity,  the  author  concludes  that  the  short  form  instrument  measures  LAN 
EUS. 

On  the  bases  of  the  reliability  and  validity  information  above,  the  null  hypothesis  is 
rejected  in  that  the  short  form  instrument  used  at  the  VA  PSHCS  to  measure  LAN  EUS  is  reliable 
and  valid.  Stated  another  way,  the  short  form  instrument  consistently  and  accurately  measures 
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Hypothesis  2  and  Demographics 

With  the  reliability  and  validity  of  the  instrument  established,  the  level  of  satisfaction  (total 
and  three  factors)  will  also  be  reliable  and  valid.  However,  before  this  discussion  pursues,  the 
demographic  data  may  help  to  better  define  the  healthcare  executive  LAN  EUS  within  the 
organization.  The  demographic  information  is  shown  in  Table  2.  Overall,  the  respondents' 
demographic  means  were  approximately  60%  male  and  40%  female  (the  cohort  was  62%/38% 
by  gender);  were  46  years  old;  have  been  using  the  LAN  1 1  to  13  months;  use  the  LAN  38%  to 
47%  of  the  workday;  approximately  70%  of  the  end  users  were  provided  with  LAN  training  by 
the  organization;  and  the  end  users  that  received  training  by  the  organization  received  the 
training  10  to  12  days  after  being  connected  to  the  LAN.  The  demographics  by  management 
level  (AO,  SASC,  and  SMS)  are  also  shown  in  Table  2.  There  were  notable  differences  between 
the  overall  demographics  and  management  level: 

a)  Senior  Management  and  Staff  were  using  the  LAN  one  to  three  months  longer  than  S  ASCs, 
and  two  to  four  months  longer  than  AOs; 

b)  Approximately  90  to  100%  of  AOs  received  LAN  training  fi-om  the  organization,  whereas 
approximately  60%  to  70%  for  SASCs  and  30%  to  50%  for  SMS; 

c)  Administrative  Officers  received  training  between  four  days  before  and  nine  days  after  the 
LAN  connection;  SASCs  received  LAN  training  between  seven  and  1 1  days  after  the  LAN 
connection;  and  SMS  received  LAN  training  12  to  25  days  after  the  LAN  connection. 

The  demographic  information  above  is  provided  to  assist  the  reader  in  developing  a 

conceptual  model  including  the  shape  and  texture  of  the  cohort  and  respondents.  These 

demographic  variables  will  be  discussed  later  in  the  third  hypothesis  section. 

With  the  establishment  of  the  short  form  instrument's  reliability  and  validity,  and  a  better 

understanding  of  the  respondents  including  the  number  of  healthcare  executives  that  received 

LAN  training,  we  can  now  discuss  the  LAN  EUS  data  obtained  from  this  study.  The  total  and 
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factor  satisfaction  scores  by  survey  and  by  management  level  are  shown  in  Tables  3  (satisfaction 
scores)  and  6  (paired  samples  test).  The  total  satisfaction  change  from  survey  one  (slightly 
satisfied  13.01)  to  survey  2  (slightly  satisfied  16.19)  was  not  significant.  However,  there  is  an 
increasing  trend  of  higher  LAN  satisfaction  among  the  healthcare  executives  at  the  VA  PSHCS. 

The  IRMS  staff  and  services  satisfaction  difference  between  survey  1  (slightly  satisfied 
1.04)  and  survey  2  (slightly  satisfied  1.30)  was  not  significant.  The  user's  LAN  knowledge  and 
involvement  satisfaction  difference  between  survey  1  (neither  dissatisfied  nor  satisfied  .52)  and 
survey  2  (neither  dissatisfied  nor  satisfied  .63)  was  not  significant.  However,  as  the  total 
satisfaction  level  increases  over  time,  so  does  the  IRMS  staff  and  services,  and  the  users' LAN 
knowledge  and  involvement  satisfaction  levels.  The  only  significant  difference  between  the 
surveys  is  the  change  of  the  information  product  factor  satisfaction  level  (survey  1,  slightly 
satisfied  1.20;  survey  2,  slightly  satisfied  1.55,  S.D.  =  .88,  t  =  2.233,  n  =  31,  30  df).  This 
suggests  that  the  healthcare  executives  are  becoming  more  satisfied  with  the  LAN  (reliability, 
relevancy,  accuracy,  precision,  and  completeness;  see  appendix  1  for  instrument  scales  and 
definitions).  Moreover,  this  information  might  be  an  indicator  for  LAN  success  and  effectiveness 
and  thereby  supports  the  implementation  of  the  LAN  infrastructure  and  desktop  enhancements. 
On  the  basis  of  the  information  provided  above,  the  null  hypothesis  is  rejected  in  that  there  is  an 
increasing  trend  of  healthcare  executive  LAN  satisfaction  over  time. 

The  total  and  three  factor  satisfaction  scores  by  management  level  are  shown  in  Table  3. 

All  but  one  of  the  satisfaction  scores  is  increasing  where  most  of  the  respondents  by  management 
level  remain  in  the  same  satisfaction  interval.  However,  there  are  a  few  noteworthy  changes  by 
interval: 

a)  The  AOs  increased  their  LAN  knowledge  and  involvement  satisfaction  interval  from  neither 
dissatisfied  nor  satisfied  to  slightly  satisfied; 
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b)  The  SMS  increased  their  IRMS  staff  and  service,  and  LAN  information  product  satisfaction 
intervals  from  neither  dissatisfied  nor  satisfied  to  slightly  satisfied; 

c)  The  SASCs'  LAN  knowledge  and  involvement  satisfaction  decreased  to  .73  from  .83  (neither 
dissatisfied  nor  satisfied),  but  remained  in  the  same  interval. 

Hypothesis  3 

There  are  several  published  LAN  satisfaction  studies  that  include  grouped  differences  (i.e., 
gender,  age,  position  held  in  the  organization,  duration  of  LAN  usage,  percent  of  time  using  the 
LAN,  and  LAN  training)  in  part,  but  not  all.  As  with  the  somewhat  new  test-retest  methodology 
being  applied  to  the  short  form  instrument,  the  grouped  differences  are  also  a  somewhat  new 
approach  beyond  the  previous  research  studies  in  relation  to  the  13  short  form  instrument  scales. 
In  this  study,  the  author  found  no  significant  findings  between  the  satisfaction  scores  and  the 
grouped  differences.  Therefore,  the  null  hypothesis  is  rejected  in  that  healthcare  executive  LAN 
EUS  at  the  VA  PSHCS  is  independent  of  grouped  differences  in  this  study.  Stated  another  way, 
the  data  suggest  that  gender,  age,  position  held  in  the  organization,  duration  of  LAN  usage, 
percent  of  time  using  the  LAN,  and  LAN  training,  is  inconclusive  regarding  the  affect  these 
variables  ha\te  on  LAN  satisfaction.  This  outcome  might  have  been  limited  due  to  the  small 
sample  size.  In  other  studies,  the  sample  sizes  were  much  larger  as  in  Nath's  (1990)  study  where 
the  sample  size  was  107;  as  in  Brent  et  al's  (1993)  study  where  the  sample  size  was  215;  and  as 
in  Smith's  (1993)  study  where  the  sample  size  was  253.  Moreover,  there  are  many  other 
challenges  other  than  the  sample  size  that  might  influence  the  subjects'  responses  between 
surveys  1  and  2.  According  to  Alan  J.B.  Anderson  (1989),  they  include,  but  are  not  limited  to: 

1 .  History — any  number  of  unknown  external  events  might  occur  and  influence  the  attitude,  and 
behavior  of  the  respondent. 

2.  Maturation — changes  within  the  respondent,  such  as  growing  older. 
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3.  Mortality — subjects  who  die  or  are  lost  to  the  study. 

4.  Observation — subjects  may  change  their  behavior,  responses,  etc.  if  they  know  that  their 
activities  are  being  observed;  this  is  also  known  as  the  Hawthorne  Effect. 

5.  Interaction  between  survey  1  and  the  LAN — subjects  might  have  completed  survey  1  and 
taken  a  training  course,  read  about  the  LAN,  etc.,  and  then  took  survey  2.  This  problem  is 
diminished  with  a  standardized  instrument  like  the  short  form  instrument. 


Summary  of  Conclusions 

The  conclusion  summary  is  provided  below  by  hypothesis. 

Hypothesis  1:  The  short  form  instrument  used  at  the  VA  PSHCS  is  a  reliable  and  valid 
measure  of  LAN  EUS. 


The  LAN  EUS  data  gathered  during  this  study  suggests: 

a)  The  short  form  instrument  is  considered  to  be  reasonably  free  of  measurement  error- 
instrument  reliability. 

b)  The  short  form  instrument  measures  LAN  EUS-validity  of  the  instrument. 

On  the  basis  of  the  reliability  and  validity  statements  above,  the  author  rejected  the  null 
hypothesis.  The  short  form  instrument  consistently  and  accurately  measured  LAN  EUS  at  the 
VA  PSHCS. 

Hypothesis  2:  Healthcare  executive  LAN  EUS  at  the  VA  PSHCS  is  expected  to  increase 
over  time. 

The  LAN  EUS  data  gathered  during  this  study  suggests: 

a)  The  VA  PSHCS  is  experiencing  an  increasing  trend  of  LAN  satisfaction  among  healthcare 
executives. 

This  suggests  that  the  healthcare  executives  are  becoming  more  satisfied  with  the  LAN 
(reliability,  relevancy,  accuracy,  precision,  and  completeness).  Moreover,  this  outcome  provides 
strong  evidence  as  an  indicator  for  LAN  success  and  effectiveness,  and  thereby  supports  the 
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implementation  of  the  LAN  infrastructure  and  desktop  enhancements.  On  the  basis  of  the 
information  provided  above,  the  null  hypothesis  is  rejected  and  the  healthcare  executives'  LAN 
satisfaction  at  the  VA  PSHCS  is  increasing  over  time. 

Hypothesis  3:  Healthcare  executive  LAN  EUS  is  independent  of  grouped  differences  (i.e., 
gender,  age,  position  held  in  the  organization,  duration  of  LAN  usage,  percent  of  time  using 
the  LAN,  and  LAN  training). 

The  LAN  EUS  data  gathered  during  this  study  suggests: 
a)  There  are  no  significant  findings  between  the  satisfaction  scores  and  the  grouped  differences. 

Therefore,  the  null  hypothesis  is  rejected  and  this  suggests  that  the  healthcare  executive 
LAN  EUS  at  the  VA  PSHCS  is  independent  of  grouped  differences.  Stated  another  way,  the 
data  suggest  that  gender,  age,  position  held  in  the  organization,  duration  of  LAN  usage,  percent 
of  time  using  the  LAN,  and  LAN  training,  is  inconclusive  regarding  the  affect  these  variables 
have  on  LAN  satisfaction. 


Recommendations 

Essentially,  there  are  several  key  recommendations  gleamed  from  this  study  in  support  of 
the  project's  objectives:  to  measure  LAN  EUS;  to  identify  opportunities  for  LAN  end  user 
training;  and  to  measure  the  IRMS  staff  relations  with  LAN  end  users.  Congruent  with  the 
objective  to  measure  LAN  EUS,  the  first  recommendation  is  to  refine  the  short  form  instrument 
or  develop  another  satisfaction  instrument  to  measure  LAN  and  other  computer  systems'  EUS  for 
system  success  and  for  other  accrediting  body  standards  such  as  from  the  Joint  Commission  on 
Accreditation  of  Healthcare  Organizations.  There  are  five  modified  questions  taken  from  the 
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short  form  instrument  that  are  essential  to  understanding  the  success  and  effectiveness  of  a 
computer  system  that  should  be  answered  to  better  meet  the  needs  of  the  VA  PSHCS's  internal 
customers: 

1 .  How  does  the  organization  know  if  the  computer  system  or  product  is  dependable  and 
consistent? 

2.  How  does  the  organization  know  if  the  computer  system  can  deliver  what  it  is  suppose  to 
deliver? 

3.  How  does  the  organization  know  if  the  computer  system  services  and  products  are  what  the 
end  users  need  to  function  in  their  present  position? 

4.  How  does  the  organization  know  if  the  computer  system  and  products  are  correct  for  the  end 
user? 

5.  How  does  the  organization  know  if  the  computer  system  and  products  meet  the  needs  of  the 
end  user? 

The  second  recommendation,  which  is  congruent  with  the  objective  to  identify 
opportunities  for  LAN  end  user  training,  is  to  continue  to  measure  LAN  BUS  among  all  staff  at 
the  VA  PSHCS  and  identify  computer  end  user  training  needs.  This  study  was  limited  to 
healthcare  executives,  which  is  only  a  small  portion  of  the  organization's  staff  and  should  be 
expanded  to  include  all  of  the  organization's  computer  systems  end  users.  Additionally,  LAN 
training  provided  to  the  respondents  ranged  from  four  days  prior  to  the  LAN  connection  and  25 
days  after  the  connection.  Furthermore,  there  appears  to  be  some  disparity  by  management  level 
regarding  the  LAN  training  provided  to  the  end  users.  Finally,  the  pattern  of  disparity  by 
management  level  also  appears  to  follow  the  implementation  of  LAN  connections.  Stated 
another  way,  the  higher  the  respondent's  position  by  management  level,  the  sooner  she  was 
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connected  to  the  LAN  and  the  least  likely  to  receive  LAN  training.  This  observation  is  not 
presented  to  state  a  fact,  but  to  suggest  further  inquiry  into  the  LAN  training  practices  at  the 
VAPSHCS. 

The  third  recommendation,  which  is  congruent  with  the  final  objective  of  this  study,  is  to 
continue  to  measure  IRMS  staff  relations  with  LAN  end  users.  Overall,  the  healthcare 
executives  in  this  study  were  slightly  satisfied  with  the  IRMS  staff  and  services.  This  outcome 
suggests  that  there  is  an  opportunity  for  improvement  that  is  congruent  with  the  continuous 
quality  improvement  cycle.  One  recommendation  is  to  use  the  average  IRMS  staff  and  services 
factor  score  as  a  starting  point,  and  have  the  IRMS  staff  answer  the  modified  five  questions 
below  that  were  taken  from  the  short  form  instrument: 

1 .  How  does  the  IRMS  employees  and  ostensible  agents  interact,  conduct,  and  associate  with 
end  users? 

2.  How  (manner  and  methods)  does  the  IRMS  employees  and  ostensible  agents  exchange 
information  with  end  users? 

3.  How  does  the  IRMS  employees  and  ostensible  agents  respond  to  the  requests  for  changes  to 
the  existing  computer  systems  or  services? 

4.  How  does  the  IRMS  employees  and  ostensible  agents  respond  to  user's  request  for  new 
applications  and  design,  development,  and  implementation  of  computer  systems?  How  long 
does  it  take  to  respond  to  a  user's  request? 

5.  How  does  the  IRMS  employees  and  ostensible  agents  respond  to  conflicting  external, 
professional  goals  and  the  goals  and  tasks  of  the  organization? 


Future  Research  with  the  Short  Form  Instrument 

As  mentioned  in  previous  studies  by  Galletta  and  Lederer  (1989),  and  by  Baroudi  and 
Orlikowski  (1988),  future  test-retest  short  form  instrument  research  is  necessary.  To  date,  no 
other  study  has  been  able  to  advance  beyond  replication  of  Baroudi  and  Orlikowski's  (1988) 
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study.  Galletta  and  Lederer  (1989)  used  test-retest  spanning  a  one-day  college  class  period. 
Therefore,  further  longitudinal  research  is  necessary  with  a  longer  period  of  time  using  the  short 
form  instrument  as  in  this  study  of  LAN  EUS. 

Another  opportunity  for  future  research  is  in  the  area  of  grouped  differences.  To  date,  no 
other  study,  except  this  project  of  LAN  EUS,  has  analyzed  the  correlation  of  grouped  differences 
(i.e.,  age,  gender,  position  held  in  the  organization,  duration  of  LAN  usage,  percent  of  time  using 
the  LAN,  and  LAN  training)  and  total  satisfaction  with  a  computer  system  using  the  short  form 
instrument.  This  could  potentially  further  the  application  of  the  short  form  instrument  from 
cross-sectional  to  longitudinal  analysis  to  provide  organizations  with  trending  capabilities. 
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Appendix 

VA  PUGET  SOUND  HEALTH  CARE  SYSTEM 
LOCAL  AREA  NETWORK 
END  USER  SATISFACTION  QUESTIONNAIRE 


The  purpose  of  this  study  is  to  measure  how  you  feel  about  certain  aspects  of  the  VA  Puget  Sound 
Health  Care  System's  Local  Area  Network  (LAN)  Services  that  are  provided  to  you  in  your  present 
position. 

On  the  following  pages  you  will  find  different  factors,  each  related  to  some  aspect  of  your  LAN  support. 
You  are  to  rate  each  factor  on  the  descriptive  scales  that  follow  it,  based  on  your  evaluation  of  the  factor. 
Your  responses  are  important  and  will  remain  anonymous.  If  you  have  any  questions,  please 
contact  Warren  Hill  at  extension  6089  at  American  Lake  Division. 


The  scale  positions  are  defined  as  follows: 

Adjective  X  : _ : _ : _ : _ : _ : _ : _ :  Adjective  Y 


(1)  (2) 

(3)  (4)  (5)  (6)  (7) 

(1) 

Extremely  X 

(5)  Slightly  Y 

(2) 

Quite  X 

(6)  Quite  Y 

(3) 

Slightly  X 

(7)  Extremely  Y 

(4) 

Neither  X  or  Y;  equally  X 

or  Y;  does  not  apply 

The  following  example  illustrates  the  scale  positions  and  flieir  meanings: 

My  vacation  in  the  Bahamas  was: 

Restful  : _ : _ : _ : _ : _ : _ :  X  :  Hectic 

(1)  (2)  (3)  (4)  (5)  (6)  (7) 

Healthy  : _ :  X  : _ : _ : _ : _ : _ :  Unhealthy 

(1)  (2)  (3)  (4)  (5)  (6)  (7) 

According  to  the  responses,  the  person's  vacation  was  extremely  hectic  and  quite  healthy. 


Reprinted  with  permission  from  M.  E.  Sharpe,  Inc., 
Publisher,  Armonk,  NY  10504 
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INSTRUCTIONS 

1 .  Check  each  scale  in  the  position  that  describes  your  evaluation  of  the  factor  being  judged. 

2.  Check  every  scale;  do  not  omit  any. 

3.  Check  only  one  position  for  each  scale. 

4.  Check  in  the  space,  not  between  spaces.  THIS,  NOT  THIS 

:  X  :  X  : 

5.  Work  rapidly.  Rely  on  your  first  impressions. 

Thank  you  for  your  cooperation. 


ANSWER  BASED  ON  YOUR  OWN  FEELINGS 
1.  Relationship  with  the  IRMS*  Staff  is: 

(the  manner  and  methods  of  interaction,  conduct,  and  association  between  the  user  and  the  IRMS 
staff) 

dissonant  : _ : _ : _ : _ : _ : _ : _ :  harmonious 

bad  : _ : _ : _ : _ : _ : _ : _ :  good 


2.  Processing  of  requests  for  changes  to  existing  LAN  systems  is: 

(the  manner,  method,  and  required  time  with  which  the  IRMS  staff  responds  to  user  requests  for 
changes  in  existing  LAN  system  or  services) 


fast  :::::::: 

slow 

untimelv  :::::::: 

timely 

Degree  of  LAN  training  provided  to  users  is: 

(the  amount  of  specialized  instruction  and  practice  that 
proficiency  in  utilizing  the  LAN  capability) 

is  afforded  to  the  user  to  increase  the  user’s 

complete  :::::::: 

incomplete 

low  :::::::: 

high 

User*s  imderstanding  of  LAN  systems  is: 

(the  degree  of  comprehension  &at  a  user  possesses  about  the  LAN  systems  or  services  that  are 
provided) 

insufficient  :::::::: 

sufficient 

complete  :::::::: 

incomplete 

User’s  feeling  of  participation  is: 

(the  degree  of  involvement  and  commitment  which  the  user  shares  with  the  IRMS  staff  and  others 
toward  the  functioning  of  the  LAN  systems  and  services) 

positive  :::::::: 

negative 

insufficient  :::::::: 

sufficient 

*IRMS  =  Information  Resources  Management  Service 


Reprinted  with  permission  from  M,  E.  Sharpe,  Inc,, 
Publisher,  Armonk,  NY  10504 
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6.  Attitude  of  the  IRMS  Staff  is: 

(the  willingness  and  commitment  if  the  IRMS  staff  to  subjugate  external,  professional  goals  in  favor 
of  organizationally  directed  goals  and  tasks) 

cooperative  : _ : _ : _ : _ : _ : _ : _ *  belligerent 

negative  : _ : _ : _ : _ : _ : _ : _ :  positive 


7.  Reliability  of  LAN  output*  is: 

(the  consistency  and  dependability  of  the  LAN  output  information) 

high  : _ : _ : _ : _ : _ : _ : _ :  low 

superior  : _ : _ : _ : _ : _ : _ : _ :  inferior 

8.  Relevancy  of  LAN  output*  is: 

(the  degree  of  congruence  between  what  the  user  wants  or  requires  and  what  is  provided  by  the  LAN 
products  or  services) 

useful  : _ : _ : _ : _ : _ : _ : _ :  useless 

relevant  : _ : _ : _ : _ : _ : _ : _ :  irrelevant 


9.  Accuracy  of  LAN  output*  is: 

(the  correctness  of  the  LAN  products  or  services) 

inaccurate  : _ : _ : _ : _ : _ : _ : _ :  accurate 

low  : _ : _ : _ : _ : _ : _ : _ :  high 


10.  Precision  of  LAN  output*  is: 

(the  variability  of  the  LAN  products  or  services  from  that  which  it  purports  to  provide) 


low  :  :  :  : 

:  :  :  ;  high 

definite  :  :  :  : 

:  :  :  :  uncertain 

1 1 .  Communication  with  the  IRMS  staff  is: 

(the  manner  and  methods  of  information  exchange  between  the  user  and  the  IRMS  staff) 

dissonant  : _ : _ : _ : _ : _ : _ : _ :  harmonious 

destructive  : _ : _ : _ : _ : _ : _ : _ :  productive 


12.  Time  required  for  new  LAN  development  is: 

(the  elapsed  time  between  the  user’s  request  for  new  applications  and  design,  development,  and/or 
implementation  of  the  application  systems  by  the  IRMS  staff) 

unreasonable  : _ : _ : _ : _ : _ : _ : _ :  reasonable 

acceptable  : _ : _ : _ : _ : _ : _ : _ :  inadequate 


*LAN  output  may  include  network  speed  and  performance  (e.g.,  log-in  time,  opening  software 
programs);  the  number  and  length  of  LAN  major  and  minor  malfunctions  that  interrupt  work; 
sharing  peripherals  (e.g.  printers);  and  sharing  information  (e.g.,  computer  files). 

Reprinted  with  permission  from  M.  E.  Sharpe,  Inc., 

Publisher,  Armonk,  NY  10504 
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13.  Completeness  of  the  LAN  output*  is: 

(the  comprehensiveness  of  the  LAN  products  or  services  content) 

sufficient  : _ : _ : _ : _ : _ : _ : _ :  insufficient 

adequate  : _ : _ : _ : _ : _ : _ i _ :  inadequate 

*LAN  output  may  include  network  speed  and  performance  (e.g.,  log-in  time,  opening  software 
programs);  the  number  and  length  of  LAN  major  and  minor  malfunctions  that  interrupt  work; 
sharing  peripherals  (e.g,  printers);  and  sharing  information  (e.g,,  computer  files). 

Reprinted  with  permission  from  M.  E.  Sharpe,  Inc., 

Publisher,  Armonk,  NY  10504 


What  is  your  present  position? 

_ Administrative  Officer 

_  Service  Chief/Program  Director/Section  Chief 

_  Senior  Management  and  Staff 

Gender:  Male _ Female _ 

Age _ 

How  many  months  have  you  been  using  the  LAN? _ 

Percent  of  time  spent  using  the  LAN? _ % 

Have  you  received  LAN  training?  Yes _ No _ 

If  yes,  how  many  days  before  or  after  you  were  connected  to  the  LAN?  Days  before _ Days  after 

If  no,  who  taught  you  how  to  use  the  LAN? _ 

Please  provide  any  brief  comments/additional  information  you  wish  to  include: 


Your  responses  will  remain  anonymous.  However,  to  correlate  your  responses  to  other  potential 
surveys,  please  provide  the  following  information: 

What  is  the  last  digit  in  the  year  that  you  graduated  from  high  school?  _ 

(e.g.  "1992"  would  equal  "2") 

What  is  the  first  letter  in  the  name  of  the  high  school  you  graduated  from? _ 

(e.g.,  "Billings  Senior  High  School"  would  equal  "B") 

What  is  the  last  letter  in  your  mother’s  maiden  name? _ 

(e.g.,  "Golden"  would  equal  "N") 

Thank  you  for  completing  this  questionnaire.  Your  responses  are  important.  Please  return  (ONLY)  the 
questionnaire  in  the  self-addressed  governmental  envelope  to  Warren  Hill  (OOH)  by  November  21, 
1997.  If  you  have  any  questions,  please  contact  Warren  Hill  at  extension  6089  at  the  American  Lake 
Division. 
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